Extreme-scale computing services over MPI: Experiences, observations and features proposal for next-generation message passing interface

The Message Passing Interface (MPI) is one of the most portable high-performance computing (HPC) programming models, with platform-optimized implementations typically delivered with new HPC systems. Therefore, for distributed services requiring portable, high-performance, user-level network access, MPI promises to be an attractive alternative to custom network portability layers, platform-specific methods, or portable but less performant interfaces such as BSD sockets. In this paper, we present our experiences in using MPI as a network transport for a large-scale distributed storage system. We discuss the features of MPI that facilitate adoption as well as aspects which require various workarounds. Based on use cases, we derive a wish-list for both MPI implementations and the MPI forum to facilitate the adoption of MPI large-scale persistent services. The proposals in the wish-list go beyond the sole needs of distributed services; we contend that they will benefit mainstream HPC applications at extreme scales as well.